A Fast Cross-Validation Algorithm for Kernel Ridge Regression by Eigenvalue Decomposition

Fast Randomized Kernel Ridge Regression with Statistical Guarantees

One approach to improving the running time of kernel-based methods is to build a small sketch of the kernel matrix and use it in lieu of the full matrix in the machine learning task of interest. Here, we describe a version of this approach that comes with running time guarantees as well as improved guarantees on its statistical performance. By extending the notion of statistical leverage scores...

Fast Relative-Error Approximation Algorithm for Ridge Regression

Ridge regression is one of the most popular and effective regularized regression methods, and one case of particular interest is that the number of features p is much larger than the number of samples n, i.e. p n. In this case, the standard optimization algorithm for ridge regression computes the optimal solution x⇤ in O(n2p + n3) time. In this paper, we propose a fast relativeerror approximati...

The Analysis of Kernel Ridge Regression Learning Algorithm

Kernel Ridge Regression via Partitioning

In this paper, we investigate a divide and conquer approach to Kernel Ridge Regression (KRR). Given n samples, the division step involves separating the points based on some underlying disjoint partition of the input space (possibly via clustering), and then computing a KRR estimate for each partition. The conquering step is simple: for each partition, we only consider its own local estimate fo...

An Identity for Kernel Ridge Regression

This paper derives an identity connecting the square loss of ridge regression in on-line mode with the loss of the retrospectively best regressor. Some corollaries about the properties of the cumulative loss of on-line ridge regression are also obtained.